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METHOD AND APPARATUS FOR FACIAL IMAGE ACQUISITON AND 
RECOGNITION 

Technical Field of the Invention 

5 The present invention relates generally to the field of image processing and 

recognition. More specifically, it relates to a method and an apparatus for face 
image acquisition and recognition where active lights are used to illuminate the 
face. 

10 Background of the Invention 

Face recognition is a biometric technology in which the technology related 
to computers, image processing, and pattern recognition is also involved to 
perform person identification based on facial images. Recently, especially after 
9.11 terror attacks, many countries in the world have attached a great importance 

15 to their public security. Accordingly, face recognition technology has been greatly 
noticed much more than ever before. 

Biometric authentication refers to a class of high tech recognition 
technologies that use human biometric traits to carry out person verification and 
identification. Biometric traits of a person, such as fingerprint, palm print, iris, 

20 deoxyribonucleic acid (DNA), are unique and stable for the individual; they 
cannot be duplicated, stolen and forgotten. Because each person's characteristics 
are distinct from others, it is possible to accurately identify a person using his/her 
unique biometrics. Existing biometric recognition methods include face 
recognition, fingerprint recognition, sound recognition, palm print recognition, 

25 signature recognition, eye iris, retina recognition and so on. 

As compared to other recognition technologies, face recognition technique is 
many advantages such that it is natural, simple and convenient, easy to operate, 
user friendly, contactless, and non-intrusive, etc. It can complete the recognition 
task without incurring much disturbance. With this technology, people no longer 
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need to worry about touching his fingerprint on the fingerprint apparatus, or 
talking to the microphone, or looking into an iris scanner. A face can be 
recognized when a person show his face to the camera. Therefore, the face 
recognition technology can be widely applied to access control, machine readable 
5 traveling documents (MRTD), e-passport, anti-terrorism, ATM, computer logon, 
safe cabinet, time attendabce, and so on. 

Typical face recognition applications include the following modes: 
Identification (1:N match) to determine a person's ID: A system (1) acquires 
the face image data, (2) extracts facial features or record from the image, (3) 

io compares it with all or part of the records of enrolled persons in database to 
calculate the similarity scores, and (4) produce a sorted list based on the 
similarity score. Finally, the system outputs the persons ID corresponding to the 
top most similarity if the top most similarity is above an acceptance threshold; 
otherwise concludes that the person is not identified. 

15 Verification (1:1 match) to verify whether the claimant. In this case, the 

system needs just to compare the facial record extracted from the image with that 
of the claimed person to give the similarity score. The system either accepts the 
claimant if the similarity score is above an acceptance threshold, or reject if 
otherwise. 

20 Surveillance: Using the techniques of face image acquisition and face 

recognition to track a person in the surveillance area and determines his location. 

Monitoring: To discover the faces in the surveillance area, far or near, 
regardless of their locations, track them and separate them from the background, 
compare the facial features with those in the database. The entire process is 
25 automatic, continuous, and real-time. 

The above application modes can be widely applied in the following 
domains: 

Personnel identification and indexing: These can be used in 
computer/network security, bank services, smart card, access control, frontier 
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control, etc; 

ID card: This can be used in voter registration, ID card, passport, driver's 
license, work identification and so on; 
Computer information safeguarding system: This uses the facial features to 
5 recognition user, safeguards the computer information; 

Crime suspect recognition system: This system stores face pictures and 
recognizes faces in analyzing incidents; 

Long-distance person identification: This is applied in surveillance, 
monitoring, CCTV, traffic control, enemy-friend recognition and so on. 
10 A face recognition process is illustrated in Fig. 1. It consists of following 

three modules: 

Image acquisition module 10: It captures face image or video images 
through image acquisition equipment (for example video camera, digital camera 
and so on), then, then sends these images or video to a computer. 
15 Feature extraction module20: Residing in a computer processor, this module 

examines the input image, detects the face, locate facial features such as eyes and 
mouth, normalize the face in pose and illumination, and extracts face features 
(face code). 

Feature matching module30: Also residing in the computer, it compares the 
20 face features extracted from the input image information (face code) with those 
stored in the database40, and find the best matched one. 

Obviously, the face feature database should be set up before the face 
recognition process. Therefore, as shown in Fig. 2, a face recognition system 
should have two main parts: Face Recognition (Part A), and Face Enrollment 
25 (Part B). Among them, the purpose of Part B is to register related personal 
information for the person to be enrolled, extract the face code of the person, and 
store the information and face code in the database for face recognition process in 
the future. 

Both enrollment and recognition (Parts A and B) include the image 
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acquisition and feature extraction modules. Of these, the face recognition part has 
an additional feature matching (comparing) module, while the face enrollment 
part has a data saving module. 

Face feature extraction process 20 is composed of several steps: face 
5 detection or tracking 201, facial feature localization and face normalization 202, 
facial feature extraction (face code generation) 203. The face detection finds the 
face in the input image or video image sequence, so that the face is separated 
from the background; the face tracking tracks detected the faces in video image 
sequence, face normalization or alignment uses localized facial landmarks (eyes 

10 and/or mouth) to normalize the geometry of the face to a standard pose and 
normalize the lighting to a standard illumination condition, face feature extraction 
calculates the face code from normalized face image. 

Face matching 30 compares the face code from the input with those of the 
enrolled persons in the database 40, one by one in turn, computes the similarity 

15 matching scores, and gives a decision for verification or identification after 
referring to a similarity threshold. 

To achieve reliable and accurate accuracy, face recognition should be 
performed based on intrinsic factors of the face only, mainly of 3D shape and 
reflectance of the facial surface. Variations brought about by extrinsic factors, 

20 including hairstyle, eyeglasses, expression, posture, and environmental lighting, 
should be eliminated. The problem of uncontrollable environmental lighting is a 
big challenge to overcome. Performance is existing face recognition system 
varies a lot under different environmental lighting. 

The research shows that: the difference of face image for same person by 

25 light change is much bigger than that of different persons. (Sees also Yael Adnin, 
Yael Moses and Shimon Ullman, "Face recognition: The problem of 
compensating for changes in illumination direction", IEEE Transactions on 
Pattern Analysis and Machine Intelligence, Vol. 19, No. 7, 1,997, pp. 712-732). 
Existing face recognition technology depends on "passive" light source, that is, 



environmental light sources. Unfortunately, in real application, the environmental 
lights vary, and are not controlled. A change in environment lights changes the 
captured face image dramatically. This in turn significantly changes extracted 
face features, and causes significant drop in recognition accuracy. 
5 Suppose for each point Pi, there is a vector = (n x ,n y ,n z )' ' , n T ; Is a unit 

vector, that is \r\ = 1 ; Assume that the light source is point source, the direction is 
s = (s x ,s y ,s 2 ), then we have the Lambertian imaging equation model, the gray 
scale Ij of Pj can be written as: 

/. = p,(x, y)n i (x, y) T • s (l) 
10 where z'=l , 2,. . k, kis the number of pixels of a face image 
p, is the surface reflection rate of Pj 
n T t indicates the surface vector it) of the point i 
• is the dot product operation 
x,y,z is the 3-D coordinate of Pj„ 
15 It can be seen from the above equation that the face image formation is 

related to the reflection and 3-D shape of the face surface, and the illumination. 
These are the three essential factors in the face image formation process. The first 
two terms are related with the intrinsic characteristic of the face itself, and also 
the important information for face recognition; the last term, illumination, is the 
20 extrinsic factor, and also the primary factor which affects face recognition 
performance. 

Although the light intensity \\s\\ also affects the gray scale of face images, 
this kind of influence can be adjusted using a simple linear transform. The 
top-most factor that affects the face recognition performance is the incidence 
25 angle of the light relative to the face surface vector. Assume that 0/ is the angle 
between the incident light ray and the face surface vector at Pi e[0,<|), the 
light intensity \s\ = 1 , then Equation (1) may be expressed as follows: 



/ . = p i ( x , y ) cos 9 1 (2) 
where, i-l,2,...,k; k is the number of pixels of a face image. 

From equation (2), we can see that when 0,- changes as a result of a change 
in the illumination direction, the face image changes accordingly. It can also be 
5 illustrated by a correlation analysis: Given two face images lighted from the left 
side and from the right side, respectively, the correlation coefficient of resulting 
images is generally a negative number; this means that the two images are 
completely different by the pixel values, even though of the same person. 

In real applications, the environment lightings generally differ from place to 
10 place, and a face recognition system has to adapt to different environmental 
lightings. However, current face recognition technology mixes both intrinsic and 
extrinsic factors in the imaging and hence cannot adapt well to the environment. 
This is why the best face recognition system can only achieve 50% accuracy (see 
also NIST 2002 Human Face Recognition Vendor Tests Evaluation Report (P.J. 
15 Phillips, P. Grother, R.J Micheals, D.M. Blackburn, E Tabassi, and J.M. Bone. 
March 2003). 

Although there are many methods for compensation and normalization of 
illumination for face recognition, they are not very effective (see: P. N. 
Belhumeur, David J. Kriegman, "What is the set of Images of an Object Under 

20 All possible Lighting Conditions?", IEEE conf. On Computer Vision and Pattern 
Recognition ", 1,996; Athinodoros S. Georghiades and Peter N. Belhumeur, 
"Illumination cone models for recognition under variable lighting: Faces ", CVPR, 
1,998; Athinodoros S. Georghiades and Peter N. Belhumeur, " From Few to many: 
Illumination cone models for face recognition under variable lighting and pose ", 

25 IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 23, No. 6, 
Pp 643-660, 2,001; Amnon Shashua, And Tammy Riklin-Raviv, "The quotient 
image: Class-based re-rendering and recognition with varying illuminations ", 
Transactions on Pattern Analysis and Machine Intelligence, Vol. 23, No. 2, 
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Ppl29-139, 2,001; T. Riklin-Raviv and A. Shashua. "The Quotient image: Class 
based recognition and synthesis under varying illumination " In Proceedings of 
the 1,999 Conference on Computer Vision and Pattern Recognition, Pages 
566--571, Fort Collins, CO, 1,999; Ravi Ramamoorthi, Pat Hanrahan, "On the 

5 relationship between radiance and irradiance: Determining the illumination from 
images of a convex Lambertian object ", J. Opt. Soc. Am., Vol. 18, No. 10, 2,001; 
Ravi Ramamoorthi, "Analytic PCA Construction for Theoretical Analysis of 
Lighting Variability in Images of a Lambertian Object", IEEE Transactions on 
Pattern Analysis and Machine Intelligence, Vol. 24, No. 10, 2002-10-21; Ravi 

10 Ramamoorthi and Pat Hanrahan, "An Efficient Representation for Irradiance 
Environment Maps", SIGGRAPH 01, Pages 497-500, 2,001; Ronen Basri, David 
Jacobs, "Lambertian Reflectance and Linear Subspaces", NEC Research Institute 
Technical Report 2000- 172R; Ronen Basri and David Jacobs, Lambertian 
Reflectance and Linear Supspaces, IEEE Transactions on Pattern Analysis and 

15 Machine Intelligence, Forthcoming; Terence Sim, Takeo Kanade, "Illuminating 
the Face", CMU-RI-TR-0 1-31, Sept. 28, 2001, etc) Among these methods, some 
requires 3-D modeling of faces, while some assumes known facial shapes. These 
limitations reduce the applicability. Moreover, the computational cost is very 
high. 

20 There have been several face recognition patents, most of them using visible 

lights and for applications. One is Chinese patent ZL99117360.X. There, it is 
about how to implement the face recognition for access control and time 
attendance, without much attention paid to face image acquisition, and influence 
of skin complexion and light changes. The recognition accurate rate of this 

25 method under the lighting changes is still low. These limit its applications. 

There have also been iris recognition techniques for accurate biometric 
identification, such as used in Iridian Corporation's products. Disadvantages of 
such technology include complexity of iris image acquisition apparatuss, and 
inconvenience of use. These limit the applications/Chinese patent ZL991 10825.6 
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has also publicized portable iris equipment. This equipment is limited by the 
similar disadvantages. 

SUMMARY OF THE INVENTION 

5 The object of the present invention is to provide a method and a apparatus 

that use active lights to illuminate the face during the acquisition of facial images; 

it can significantly reduce or even eliminate unfavorable influence of variable 

environmental lightings, so as to acquire facial images of a good illumination 

condition and to achieve high recognition rate. 
10 A further object of the invention is to provide a method and a apparatus for 

facial image recognition, where the face localization and recognition are done in 

a fast way according to accurate and fast localization of eyes in the acquired face 

images; it leads to high recognition accuracy without the need to set up 

environmental lights, being practical and convenient for applications. 
15 A still further object of the invention is to provide a system that uses near 

infrared lighting for achieving reliable and accurate face recognition; the system 

is of low cost, highly compact, and highly automatic. 

To accomplish the above objects, the present invention provides a face 

recognition method, comprising the following steps: 
20 starting a face recognition apparatus ; 

providing an active lights to illuminate a target face when an user 

approaches said face recognition apparatus; 

providing an image acquisition unit to capture a plurality of images from a 

target face illuminated by an active lights; 
25 sending at least one facial image acquired by said image capturing unit to a 

data processing unit, and detecting and/or localizing a positions of eyes and/or 

said face by said data processing unit; 

cropping a portion of said facial image and extracting facial feature from 

said portion of said facial image by said data processing unit; 
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comparing facial feature with that of previously extracted and stored in a 
face database; 

outputting a recognition result obtained from said comparing step. 

According to the invention, there is provided a face recognition method, 
5 wherein said active light is a near infrared light sources, or visible light sources , 
or flash lights, or a combination of them 0 

According to the invention, there is provided a face recognition method, 
wherein a total energy of an active lighting and a environmental lighting on said 
face area is greater than that of environmental lighting. 
10 According to the invention, there is provided a face recognition method, 

wherein a total energy of said active lighting and said environmental lighting on a 
face area is not lower than twice that of environmental lighting. 

According to the invention, there is provided a face recognition method, 
wherein said method further includes a step to judge whether the detection of face 
15 and eyes is successful; if successful, continue with next step ; otherwise, go back 
to acquire a new face image, in a process of detecting a face and/or localizing eye 
positions. 

According to the invention, there is provided a face recognition method, 
wherein said method further includes a step that it could chose to localize the eye 
20 positions by localizing the specularity (highlight spot) in each of the eyes. 

According to the invention, there is provided a face recognition method, 
wherein said method further includes a step that said image capturing unit can 
track said face area illuminated by an active lights. 

To better accomplish the above object, the present invention also provide a 
25 facial image acquisition method, comprising the following steps: 

Providing a plurality of active lighting to illuminate a face area, 
Providing an image capturing unit for capturing a facial image of a target 
face, and sending said facial image to a data processing unit used for localizing 
and recognizing said target face; 
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Wherein a total energy of said active lighting and said environmental 
lighting on said face area is greater than that of environmental lighting. 

According to the invention, there is provided a facial image acquisition 
method, wherein said total energy of said active lighting and said environmental 
5 lighting on said face area is not lower than twice that of environmental lighting. 

According to the invention, there is provided a facial image acquisition 
method, wherein an angle between the direction of an active light and an axis of a 
camera lens is between 0° to 90°, with a optimal angle being 0°,and said active 
lighting and said camera lens is relatively fixed . 
10 According to the invention, there is provided a facial image acquisition 

method, wherein said active lighting is made of near infrared light sources , or 
visible light sources , or flash lights, or a combination of them 0 

According to the invention, there is provided a facial image acquisition 
method, wherein said data processing unit can make use of the specularity in each 
15 of the eyes to localize the eye position, after a facial image is captured. 

To better accomplish the above object, the present invention also provides a 
facial image acquisition apparatus used for realizing a facial image acquisition 
method, wherein comprising an active light, an image capturing unit, a power 
switch and a data processing unit; 
20 Said active lights used for illuminating a face area; 

Said power switch use for controlling said active lights to illuminate said 
face area; 

Said image capturing unit used for capturing facial images of said face area, 
and sending at least one facial image to said data processing unit; 
25 Said data processing unit used for receiving images from said image 

capturing unit, and localizing eyes and face in said facial image, cropping a 
portion of said facial image, and extracting facial features, and comparing facial 
features with that of previously extracted and stored in a facial image database. 
According to the invention, there is provided a facial recognition apparatus, 
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wherein a total energy of said active lighting and said environmental lighting on 
said face area is greater than that of environmental lighting. 

According to the invention, there is provided a facial recognition apparatus, 
wherein a position of said active lighting and said image capturing unit is 
5 relatively fixed, and a angle between a direction of said active lighting and a axis 
of the camera lens of said image apparatus between 0° to 90°. 

According to the invention, there is provided a facial recognition apparatus, 
wherein the direction of active lighting is approximately parallel to axis of a 
camera lens. 

10 According to the invention, there is provided a facial recognition apparatus , 

wherein said active lighting is made of near infrared light sources , or visible light 
sources , or flash lights, or a combination of them„ 

According to the invention, there is provided a facial recognition apparatus, 
wherein a near infrared lights has a wavelength between 740nm-4000nm, or a 

15 plurality of several wavelengths. 

According to the invention, there is provided a facial recognition apparatus, 
wherein an infrared filter is disposed on an infrared camera lens for cutting off 
visible lights radiation while allowing near infrared light radiation to pass 
through. 

20 According to the invention, there is provided a facial recognition apparatus, 

wherein said infrared filter is of band-pass or long-pass, so as to suppress visible 
lights while allowing an infrared rays to pass. 

According to the invention, there is provided a facial recognition apparatus, 
wherein there is a display device for displaying facial image, used for adjusting 
25 the position of a target face in vertical and horizontal directions. 

According to the invention, there is provided a facial recognition apparatus, 
wherein said display device is a mirror or a liquid crystal apparatus (LCD). 

According to the invention, there is provided a facial recognition apparatus , 
wherein said image capturing unit is a video camera or a digital camera. 



According to the invention, there is provided a facial recognition apparatus , 
wherein said data processing unit comprises a PC / computer or an embedded 
processor in which image processing software is installed. 

According to the invention, there is provided a facial recognition apparatus, 
5 wherein said power switch is a proximity sensor switch or an RFID controlled 
switch. 

According to the invention, there is provided a facial recognition apparatus, 
wherein said active lighting is mounted around lens of said image capturing unit. 
The present invention can effectively reduce the unfavorable impact of 
10 environmental lighting on face images, so as to achieve high accuracy under 
different environments; when in use, the face recognition apparatus uses active 
lighting to illuminate the face area, with the strength of the active lighting greater 
than that of environmental lighting, and with the relative position and direction 
between the active lights and the camera fixed; these give the best possible face 
15 images in order to achieve the best recognition results. 

BRIEF DESCRIPTION OF THE DRAWINGS 

FIG.1 is a schematic diagram of a face recognition process; 
FIG.2 is a schematic flowchart diagram including both face recognition and 
20 enrollment processes; 

FIG.3 is a schematic illustration of a angle between an active light direction 
and camera lens axis; 

FIG.4 is a schematic illustration of an exemplar system that embodies a 
face recognition method in the present invention; 
25 FIG.4a is a procedure for an embodiment of a face recognition method in 

Fig.4; 

FIG.4b is a diagram of an image acquisition and data processing modules 
for a system in Fig.4; 

FIG. 5 illustrates specular highlight spots in eyes caused by active lighting; 
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FIG.6 is a schematic diagram of a imaging camera with active lights; 

FIG. 7 is a schematic illustration of an access control system with the present 
invention of face recognition method incorporated; 

FIG. 8 is a schematic illustration of an application of the present invention of 
5 face recognition method in machine readable travel document (MRTD); 

FIG. 8a is a schematic diagram of a face image acquisition in the face 
recognition based MRTD system in Fig. 8; 

FIG. 8b is a schematic diagram of a face recognition in a face recognition 
based MRTD system in Fig.8. 

10 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 

Detailed embodiments of the present invention are disclosed herein, with an 
illustrative drawings and an exemplar embodiment: 

Fig.4 discloses a preferred embodiment of an imaging system including 

15 image acquisition apparatus and/or image recognition apparatus according to the 
present invention, comprising active lights (LED) 421, camera 422, mirror (as an 
aid for face positioning) 423, optical filter 424, control switch 426, data 
processing unit 430, indicator LED, and power supply; an active light LED are 
evenly distributed around the camera422, and in the middle are the mirror 423, 

20 the filter 424 and the camera 422; the mirror 423 is in the middle of the box of 
the imaging system, in the middle of the mirror is the filter 424 and the camera 
422; the mirror 424 is inside or in frontal of the camera lens. The camera is 
connected electronically to the data processing unit. The control switch 426 is a 
infrared sensor switch, located in the lower part of the imaging box. an indicator 

25 illuminator is located above the camera 422. The control switch 426 is connected 
to the active lights 421, the camera 422, illuminator 425, and the power supply; 
when an infrared sensor in the switch 426 is triggered on, the switch 426 turns on 
the active lights 421 and the camera 422, and the illuminator 425 turns red and 
blinking, meaning active lights and the camera are working; when the switch 426 
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turns off, the active lights 421 and the camera 422 stop, and the illuminator turns 
green, meaning standby. 

First, the active lights 421 illuminate on the face area 410, the camera 422 
(which can be a web camera, a CCTV camera, or specialized infrared camera) 
5 captures an image of the face 410; the acquired image is transmitted to the data 
processing unit where face image recognition takes place. 

Fig.4a reveals an embodiment of a face recognition system given in the 
present invention, including the following steps: 

Step 100, start a face image acquisition system 420; 
10 Step 110, when human body approaches the system 420, an infrared sensor 

is triggered on, and the active lights 421 illuminate the face area; 

Step 120, the camera 422 captures images of the face area illuminated by 
the active lights 421; 

Step 130, the camera 422 sends at least one face image to the data 
15 processing unit (such as a PC or an embedded data processing unit) 430; 

Step 140, the data processing unit 430 finds the face from the image and 
locates the positions of the eyes and/or face; 

Step 150, if the eye/face localization is successful, execute step 160; 
Otherwise, execute step 130; 
20 Step 1 60, crop the face area from the image; 

Step 170, extract facial feature template; 

Step 1 80, compare the extracted facial feature template with those stored in 
the face template database; 

Step 190, output recognition result. 
25 In the above steps, the total energy of the active lighting 421 and the 

environmental lighting 427 on the face area is greater than twice that of 
environmental lighting. For example, if the strength of the environment lighting 
is 30 LUX, and that of the active lighting is 120LUX, then the strength of the 
active lighting is 4 times that of the environmental lighting. 
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Generally, an active lights in the present invention can include infrared 
lights, flash lights, and/or visible lights. When a flash light is used, the strength of 
the flash is much greater than that of environmental lights, hence the influence of 
the latter is much reduced. Similar effect can be achieved using visible lights. 
5 In Fig.4 and Fig.4a, the active lights 421 can be infrared lights. Because 

human eyes are insensitive to lights in the infrared spectrum, the infrared types of 
active lights causes minimum disturbance to the human; meanwhile, an infrared 
optimal filter 412 can be added to the camera lens, to cut off visible lights in the 
environmental lighting, so as to further reduce the influence of environmental 

10 lighting; therefore, an infrared lighting is the most suitable type of active lighting. 

In any embodiment of the present invention, whatever type of active lights 
are used to illuminate the face, the relative position between the active lights and 
the camera should be relatively fixed, and the angle between the direction of the 
active lighting and the axis of the camera lens should be in a sharp angle. 

15 Refer to Fig.4. During the enrollment and recognition processes, the relative 

position between the face 410 and the camera 422 should not be changed, and the 
face plane and the axis of the camera 422 should be perpendicular to each other 
(i.e. the vector normal to the facial plane should be parallel to the axis of the 
camera); as such, the angle 0 between the normal vector and the camera axis is 

20 relatively unchanged, and the resulting image is most stable under the active 
lighting. 

When infrared lighting is used, an infrared optical filter can be mounted on 
the camera lens, so as to cut off the shorter wavelength visible lights, and to 
further reduce the influence of environmental lights. For the present invention, 
25 the preferred infrared lights are of near infrared in the wavelength range of 
740nm-1700nm, or of mid infrared in the wavelength range of 1700nm-4000nm. 

When an infrared optical filter is used, the filter can be either band-pass or 
long-pass type. For example, when the infrared lights are 850nm LEDs, a 
band-pass filter could be chosen, such that it has the central wavelength of 850nm 
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to allow infrared ray of around 850nm to pass while cutting of ray of wavelengths 
shorter than 800nm and longer than 900nm; or a long-pass filter could be chosen, 
such that it allows infrared ray of wavelength longer than 800nm to pass, while 
cutting off ray of wavelengths shorter than 800nm. 
5 In Fig.4 and Fig.4b, a data processing unit430 in the present invention can 

be one of PC or an embedded image data processing unit (cf Fig.4b). 

In Fig.4b, to simplify the apparatus, one could integrate all components into 
one circuit board and install the board in a casing box; the board circuits include 
the infrared sensor switch 426, analog comparator 4223, single-chip 

10 microcomputer 4222, camera 422 (eg LogiTech Pro4000), control pecker 4221, 
active lights 421 (near infrared LED array), and imbedded image data processing 
unit 430 (eg MCS-51 series). 

In Fig.5a and Fig.5b, one could make use of the specular highlights in the 
eyes (Fig.5a) for the eye and face localization, which is an effective and 

15 computationally efficient means. The active infrared lights cause a specular 
highlight in an eye, which can be seen in the face image. Therefore, one can 
detect the eyes and the face by detecting the highlights in the eyes. After the two 
highlights in the eyes are detected, one can locate the face area according to the 
geometric relationship between the two eyes and that between the eyes and the 

20 face. This enables fast and accurate face localization and much simplifies the face 
detection problem. 

Refer to Fig.3 again. Let the angle between the active light direction and the 
camera axis be 9, environmental light be Si and active light be S 2 , then the 
aformentioned equation (1) can be written as 

25 I t = Pi(x, y)n i (x, y) T • (s i + s 2 ) (3) 

where i=l,2,...,k; 

If the strength of the active lighting Si is much greater than that of the 
environmental lighting S 2 , ie || 1 S , 1 ||»||5 2 ||, then equation (3) can be approximated 
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by 

/, * P,{x, y)n,(x, y) T • s l (4) 
where i=l,2,...,k; 

If in the process of face recognition, a further constraint is imposed, namely, 
the relative position between the face and the camera is un-changed and so is the 
angle between the facial surface normal and active light direction, then according 
to equation (4), the acquired image is determined by the intrinsic properties of the 
face (ie, facial surface albedo and facial surface normal), nearly regardless of 
environmental lighting. Facial images acquired in such as way is most stable 
and best for face recognition. 

Applications 

Fig.6 and Fig.7 disclose an embodiment of the present invention for face 
recognition based access control. 

Refer to Fig.7. On a door 400 is an access controller 450. The active light 
image acquisition system 420 transmits the face image to the image data 
processing unit 430, the process 430 makes a decision of access permission, and 
send the decision to the controller 450 to grant or deny the access, 

In Fig.6 and Fig.7, the imaging system 420 includes 8-12 infrared LEDs of 
wavelength 850nm. The LEDs are mounted in frontal of the camera, in co-axis to 
the camera lens (the angle is 0 degree when the facial plane is perpendicular to 
the active light direction). With the 850nm band-pass infrared filter 423, the ray 
of 850nm LEDs can pass through the filter, whereas ray of other wavelength is 
cut off. Or a long-pass filter may be used to allow ray of wavelength above 
800nm to pass while cutting off ray below 800nm. The camera captures images 
of the face 410, and sends them to the image data processing unit 430. Making 
use of the specular highlights, the processor detects the positions of the eyes and 
hence that of the face; the pose of the face is then corrected, and facial feature 



template extracted and compared; a recognition decision is made. The data 
processing unit then sends a signal to the controller according to the decision 
result, to control the access of the door. In this embodiment, the image data 
processing unit is a desktop PC. 
5 Figs. 8, 8a and 8b disclose another embodiment of the present invention for 

face biometric based machine readable travel document (MRTD) . The first phase 
is face image enrollment, shown in Fig.8a, including the following major steps: 
Step 300, start an image enrollment system; 

Step 310, the passenger hands in the travel document 502 when the body 
10 approaches to within about 50cm from the counter 500. The infrared sensor 
switch turns on the active lights (near infrared LEDs) to illuminate the face area; 

Step 320, the passenger moves his head so that he can see his face in the 
middle of the mirror, so that the active light camera with an optical filter can take 
pictures of the face; 

15 Step 330, the camera captures at least one image and send it to the image 

data processing unit (or a PC); 

Step 340, the data processing unit locates the two highlight spots from the 
image; 

Step 350, if two highlights are detected, execute S3 60, otherwise, execute 
20 S330; 

Step 360, crop the face area from the image, based on the two detected 
highlight spots; 

Step 370, extract facial feature template(s); 

Step 380, store the extracted facial template(s). 
25 Fig. 8b discloses further details of face image acquisition and processing, 

including the following steps: 

Step 200, start a face recognition system; 

Step S210, the passenger hands in the travel document 502 when the body 
approaches to within about 50cm from the counter 500. The infrared sensor 
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switch turns on the active lights (near infrared LEDs) to illuminate the face area; 

Step 220, the passenger moves his head so that he can see his face in the 
middle of the mirror, so that the active light camera with an optical filter can take 
pictures of the face; 

5 Step 230, the camera captures at least one image and send it to the image 

data processing unit (or a PC); 

Step 240, the data processing unit locates the two highlight spots from the 
image; 

Step 250, if two highlights are detected, execute S3 60, otherwise, execute 
10 S230; 

Step 260, crop the face area from the image, based on the two detected 
highlight spots; 

Step 270, extract facial feature template; 

Step 280, compare the extracted facial template with those stored in the 
15 database; 

Step 290, output recognition result. 

In real applications, the face enrollment system and the face recognition 
system can be built into one combined system. The difference is that the latter 
does not include the enrollment phase. The custom inspector checks the 
20 documents against the enrolled passenger, associate the personal information with 
the enrolled facial image, and test whether the person can be verified his identity 
successfully by the system. 

In the embodiment shown in Fig.8, the mirror can be replaced by an LCD 
display, so that the user can adjust the head position according to the feedback 
25 image shown on LCD. One may use a digital camera type apparatus as a image 
capturing unit and also use it as the display. 

Further, the imaging system of the present invention can be on a motion 
platform, to be an elevator-pan-tilt-zoom camera unit. Such a apparatus can track 
the people, control the active lights, and capture face images. It also caters for 
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people of different heights. 

The present invention can enable face recognition in the complete darkness 
without environmental lighting. 

The present has further advantages such as being highly accurate and stable, 
5 compact, low in cost, autonomous, convenient to use in various applications and 
for installation and maintenance. 

New characteristics and advantages of the invention covered by this 
document have been set forth in the foregoing description. It will be understood, 
however, that this disclosure is, in many respects, only illustrative. Changes may 
10 be made in details, particularly in matters of shape, size, and arrangement of parts, 
without exceeding the scope of the invention. The scope of the invention is, of 
course, defined in the language in which the appended claims are expressed. 
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